System and method for interactive 360-degree video creation

ABSTRACT

A method is provided for creating a single interactive video file. The method includes receiving a selection of a plurality of 360-degree videos including a first 360-degree video and a second 360-degree video. The method also includes linking the first 360-degree video to the second 360-degree video to enable transitioning between the first 360-degree video and the second 360-degree video. The method also includes compiling the plurality of 360-degree videos into the single interactive file. The single interactive video file includes an index used to identify the second 360-degree video within the single interactive video file when the link is executed.

CROSS-REFERENCE TO RELATED APPLICATION(S) AND CLAIM OF PRIORITY

This application claims priority under 35 U.S.C. §119(e) to U.S.Provisional Patent Application No. 62/208,311 filed on Aug. 21, 2015,entitled “INTERACTIVE 360 VIDEO PROCESS.” The content of theabove-identified patent document is incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates generally to video creation. More specifically,this disclosure relates to a system and method for interactive 360-videocreation.

BACKGROUND

In virtual reality headsets there is a new medium of video that is verypopular, 360-degree video. Now that resolutions and processing power isincreasing, 360-degree video experiences are more prevalent. Whenwatching a 360 video though, the user feels immersed, but is unable tointeract with the scene.

SUMMARY

Embodiments of the present disclosure a system and method forinteractive 360-video creation.

In one example embodiment, a method is provided for creating a singleinteractive video file. The method includes receiving a selection of aplurality of 360-degree videos including a first 360-degree video and asecond 360-degree video. The method also includes link the first360-degree video to the second 360-degree video to enable transitioningbetween the first 360-degree video and the second 360-degree video. Themethod also includes compiling the plurality of 360-degree videos intothe single interactive file. The single interactive video file includesan index used to identify the second 360-degree video within the singleinteractive video file when the link is executed.

In another example embodiment, an apparatus is provided for creating asingle interactive video file. The apparatus includes a memory elementconfigured to store the single interactive video file. The apparatusalso includes at least one processor coupled to the memory element. Theprocessor is configured to receive a selection of a plurality of360-degree videos including a first 360-degree video and a second360-degree video. The processor is also configured to link the first360-degree video to the second 360-degree video to enable transitioningbetween the first 360-degree video and the second 360-degree video. Theprocessor is also configured to compile the plurality of 360-degreevideos into the single interactive file. The single interactive videofile includes an index used to identify the second 360-degree videowithin the single interactive video file when the link is executed.

In yet another example embodiment, a non-transitory computer-readablemedium is provided. The computer-readable medium comprises program codefor creating a single interactive video file. The program code, whenexecuted by at least one processor, causes the electronic device toreceive a selection of a plurality of 360-degree videos including afirst 360-degree video and a second 360-degree video. The program code,when executed by at least one processor, also causes the electronicdevice to linking the first 360-degree video to the second 360-degreevideo to enable transitioning between the first 360-degree video and thesecond 360-degree video. The program code, when executed by at least oneprocessor, also causes the electronic device to compile the plurality of360-degree videos into the single interactive file. The singleinteractive video file includes an index used to identify the second360-degree video within the single interactive video file when the linkis executed.

Other technical features may be readily apparent to one skilled in theart from the following figures, descriptions, and claims.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words and phrases usedthroughout this patent document. The term “couple” and its derivativesrefer to any direct or indirect communication between two or moreelements, whether or not those elements are in physical contact with oneanother. The terms “transmit,” “receive,” and “communicate,” as well asderivatives thereof, encompass both direct and indirect communication.The terms “include” and “comprise,” as well as derivatives thereof, meaninclusion without limitation. The term “or” is inclusive, meaningand/or. The phrase “associated with,” as well as derivatives thereof,means to include, be included within, interconnect with, contain, becontained within, connect to or with, couple to or with, be communicablewith, cooperate with, interleave, juxtapose, be proximate to, be boundto or with, have, have a property of, have a relationship to or with, orthe like. The term “controller” means any device, system or part thereofthat controls at least one operation. Such a controller may beimplemented in hardware or a combination of hardware and software and/orfirmware. The functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely. Thephrase “at least one of,” when used with a list of items, means thatdifferent combinations of one or more of the listed items may be used,and only one item in the list may be needed. For example, “at least oneof: A, B, and C” includes any of the following combinations: A, B, C, Aand B, A and C, B and C, and A and B and C.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is formed fromcomputer readable program code and embodied in a computer-readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer-readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitorycomputer-readable medium includes media where data can be permanentlystored and media where data can be stored and later overwritten, such asa rewritable optical disc or an erasable memory device.

Definitions for other certain words and phrases are provided throughoutthis patent document. Those of ordinary skill in the art shouldunderstand that in many if not most instances, such definitions apply toprior as well as future uses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, in which like referencenumerals represent like parts:

FIG. 1 illustrates an example computing system in which one or moreembodiments of the present disclosure may be implemented;

FIGS. 2 and 3 illustrate example devices in a computing system in whichone or more embodiments of the present disclosure may be implemented;

FIG. 4 illustrates a mapping for a plurality of 360-degree videos inaccordance with an embodiment of this disclosure;

FIG. 5 illustrates a portion of a 360-degree video with a hotspot 504 inaccordance with an embodiment of this disclosure;

FIG. 6 illustrates a single interactive video file compiled from aplurality of 360-degree videos in accordance with an embodiment of thisdisclosure;

FIG. 7 illustrates a pictorial representation of a video-editing tool inaccordance with an embodiment of this disclosure;

FIG. 8 illustrates a pictorial representation of selecting hotspots invideo-editing tool 700 in accordance with an embodiment of thisdisclosure; and

FIG. 9 illustrates a process for creating a single interactive videofile according to embodiments of the present disclosure.

DETAILED DESCRIPTION

FIGS. 1 through 9, discussed below, and the various embodiments used todescribe the principles of the present disclosure in this patentdocument are by way of illustration only and should not be construed inany way to limit the scope of the disclosure. Those skilled in the artwill understand that the principles of the present disclosure may beimplemented in any suitably-arranged system or device.

FIG. 1 illustrates an example computing system 100 according to thisdisclosure. The embodiment of the computing system 100 shown in FIG. 1is for illustration only. Other embodiments of the computing system 100could be used without departing from the scope of this disclosure.

As shown in FIG. 1, the system 100 includes a network 102, whichfacilitates communication between various components in the system 100.For example, the network 102 may communicate Internet Protocol (IP)packets, frame relay frames, Asynchronous Transfer Mode (ATM) cells, orother information between network addresses. The network 102 may includeone or more local area networks (LANs), metropolitan area networks(MANs), wide area networks (WANs), all or a portion of a global networksuch as the Internet, or any other communication system or systems atone or more locations.

The network 102 facilitates communications between at least one server104 and various client devices 106-114. Each server 104 includes anysuitable computing or processing device that can provide computingservices for one or more client devices. Each server 104 could, forexample, include one or more processing devices, one or more memoriesstoring instructions and data, and one or more network interfacesfacilitating communication over the network 102.

Each client device 106-114 represents any suitable computing orprocessing device that interacts with at least one server or othercomputing device(s) over the network 102. In this example, the clientdevices 106-114 include a desktop computer 106, a mobile telephone orsmartphone 108, a personal digital assistant (PDA) 110, a laptopcomputer 112, and a tablet computer 114. However, any other oradditional client devices could be used in the computing system 100.

In this example, some client devices 108-114 communicate indirectly withthe network 102. For example, the client devices 108-110 communicate viaone or more base stations 116, such as cellular base stations oreNodeBs. Also, the client devices 112-114 communicate via one or morewireless access points 118, such as IEEE 802.11 wireless access points.Note that these are for illustration only and that each client devicecould communicate directly with the network 102 or indirectly with thenetwork 102 via any suitable intermediate device(s) or network(s).

As described in more detail below, one or more of the client devices108-114 can access a video editing tool on a server 104 to compilemultiple videos into a single interactive video file. The videos can belocated on the client devices 108-114 and uploaded to the server 104 forediting, the videos may already be stored on the server 104, or acombination thereof. In a different example embodiment, the videoediting tool can be located on one of the client devices 108-114. In yetother embodiments, multiple client devices 108-114 can jointly accessthe server 104 to add videos for compiling into the single interactivevideo.

An embodiment of this disclosure provides for the single interactivevideo file to be stored on the server 104, or another server, andaccessed by one of the client devices 108-114 for playing by a videoplayer. In another embodiment, the single interactive video file can bedownloaded and stored on the client device.

Although FIG. 1 illustrates one example of a computing system 100,various changes may be made to FIG. 1. For example, the system 100 couldinclude any number of each component in any suitable arrangement. Ingeneral, computing and communication systems come in a wide variety ofconfigurations, and FIG. 1 does not limit the scope of this disclosureto any particular configuration. While FIG. 1 illustrates oneoperational environment in which various features disclosed in thispatent document can be used, these features could be used in any othersuitable system.

FIGS. 2 and 3 illustrate example devices in a computing system accordingto this disclosure. In particular, FIG. 2 illustrates an example server200, and FIG. 3 illustrates an example UE 300. The server 200 couldrepresent the server 104 in FIG. 1, and the UE 300 could represent oneor more of the client devices 106-114 in FIG. 1.

As shown in FIG. 2, the server 200 includes a bus system 205, whichsupports communication between at least one processing device 210, atleast one storage device 215, at least one communications unit 220, andat least one input/output (I/O) unit 225.

The processing device 210 executes instructions that may be loaded intoa memory 230. The processing device 210 may include any suitablenumber(s) and type(s) of processors or other devices in any suitablearrangement. Example types of processing devices 210 includemicroprocessors, microcontrollers, digital signal processors, fieldprogrammable gate arrays, application specific integrated circuits, anddiscreet circuitry.

The memory 230 and a persistent storage 235 are examples of storagedevices 215, which represent any structure(s) capable of storing andfacilitating retrieval of information (such as data, program code,and/or other suitable information on a temporary or permanent basis).The memory 230 may represent a random access memory or any othersuitable volatile or non-volatile storage device(s). The persistentstorage 235 may contain one or more components or devices supportinglonger-term storage of data, such as a ready only memory, hard drive,Flash memory, or optical disc.

The communications unit 220 supports communications with other systemsor devices. For example, the communications unit 220 could include anetwork interface card or a wireless transceiver facilitatingcommunications over the network 102. The communications unit 220 maysupport communications through any suitable physical or wirelesscommunication link(s).

The I/O unit 225 allows for input and output of data. For example, theI/O unit 225 may provide a connection for user input through a keyboard,mouse, keypad, touchscreen, or other suitable input device. The I/O unit225 may also send output to a display, printer, or other suitable outputdevice.

Note that while FIG. 2 is described as representing the server 104 ofFIG. 1, the same or similar structure could be used in one or more ofthe client devices 106-114. For example, a laptop or desktop computercould have the same or similar structure as that shown in FIG. 2. In oneor more embodiments, when the server 104 is a laptop or desktopcomputer, the video editing tool may exist on the computer for use by auser in compiling one or more videos. The compiled single interactivevideo file can then later be uploaded to another server, client device,and/or computer.

As described in more detail below, embodiments of the present disclosureprovide a tool set that allows users to create a project and upload a360-degree video or multiple 360-degree videos to a project, and thendefine hotspots overlaid on the 360-degree videos as points of action orinteraction. In one example, the tool is web based and located on server200. In another example, the tool is a downloadable application for acomputer, such as when server 200 is used as the desktop computer 106 orlaptop computer 112, or mobile telephone or smartphone 108.

FIG. 3 illustrates user equipment (UE) 300 in which one or moreembodiments of the present disclosure may be implemented. The embodimentof the UE 300 illustrated in FIG. 3 is for illustration only, the UE 300comes in a wide variety of configurations, and FIG. 3 does not limit thescope of this disclosure to any particular implementation of a UE.

In various embodiments, the UE 300 may take different forms, and thepresent disclosure is not limited to any particular form. For example,the UE 300 may be a mobile communication device, such as, for example, amobile station, head mountable display, a subscriber station, a wirelessterminal, a smart phone, a tablet, etc., that is useable with 360-degreevideo, virtual reality (VR), and/or augmented reality (AR) applications.In other examples, the UE 300 may include a headset and take the form ofa wearable electronic device, such as, for example, glasses, goggles, ahelmet, etc., for the VR and/or AR applications.

As shown in FIG. 3, the UE 300 includes an antenna 305, a radiofrequency (RF) transceiver 310, transmit (TX) processing circuitry 315,a microphone 320, and receive (RX) processing circuitry 325. The UE 300also includes a speaker 330, a processor 340, an input/output (I/O)interface (IF) 345, a touchscreen 350, a display 355, a memory 360, andone or more sensors 365. The memory 360 includes an operating system(OS) 361 and one or more applications 362.

The RF transceiver 310 receives, from the antenna 305, an incoming RFsignal transmitted by an access point (e.g., base station, Wi-Fi router,Bluetooth device) for a network (e.g., a Wi-Fi, Bluetooth, cellular, 5G,LTE, LTE-A, WiMAX, or any other type of wireless network). The RFtransceiver 310 down-converts the incoming RF signal to generate anintermediate frequency (IF) or baseband signal. The IF or basebandsignal is sent to the RX processing circuitry 325, which generates aprocessed baseband signal by filtering, decoding, and/or digitizing thebaseband or IF signal. The RX processing circuitry 325 transmits theprocessed baseband signal to the speaker 330 (such as for voice data) orto the processor 340 for further processing (such as for web browsingdata).

The TX processing circuitry 315 receives analog or digital voice datafrom the microphone 320 or other outgoing baseband data (such as webdata, e-mail, or interactive video game data) from the processor 340.The TX processing circuitry 315 encodes, multiplexes, and/or digitizesthe outgoing baseband data to generate a processed baseband or IFsignal. The RF transceiver 310 receives the outgoing processed basebandor IF signal from the TX processing circuitry 315 and up-converts thebaseband or IF signal to an RF signal that is transmitted via theantenna 305.

The processor 340 can include one or more processors or other processingdevices and execute the OS 361 stored in the memory 360 in order tocontrol the overall operation of the UE 300. For example, the processor340 could control the reception of forward channel signals and thetransmission of reverse channel signals by the RF transceiver 310, theRX processing circuitry 325, and the TX processing circuitry 315 inaccordance with well-known principles. In some embodiments, theprocessor 340 includes at least one microprocessor or microcontroller.

The processor 340 is also capable of executing other processes andprograms resident in the memory 360. The processor 340 can move datainto or out of the memory 360 as required by an executing process. Insome embodiments, the processor 340 is configured to execute theapplications 362 based on the OS 361 or in response to signals receivedfrom eNBs or an operator. The processor 340 is also coupled to the I/Ointerface 345, which provides the UE 300 with the ability to connect toother devices, such as laptop computers and handheld computers. The I/Ointerface 345 is the communication path between these accessories andthe processor 340.

The processor 340 is also coupled to the touchscreen 350 and the display355. The operator of the UE 300 can use the touchscreen 350 to enterdata and/or inputs into the UE 300. The display 355 may be a liquidcrystal display, light-emitting diode (LED) display, optical LED (OLED),active matrix OLED (AMOLED), or other display capable of rendering textand/or graphics, such as from websites, videos, games, etc.

The memory 360 is coupled to the processor 340. Part of the memory 360could include a random access memory (RAM), and another part of thememory 360 could include a Flash memory or other read-only memory (ROM).

UE 300 further includes one or more sensors 365 that can meter aphysical quantity or detect an activation state of the UE 300 andconvert metered or detected information into an electrical signal. Forexample, sensor 365 may include one or more buttons for touch input,e.g., on the headset or the UE 300, a camera, a gesture sensor, agyroscope or gyro sensor, an air pressure sensor, a magnetic sensor ormagnetometer, an acceleration sensor or accelerometer, a grip sensor, aproximity sensor, a color sensor 165H (e.g., a Red Green Blue (RGB)sensor), a bio-physical sensor, a temperature/humidity sensor, anillumination sensor 165K, an Ultraviolet (UV) sensor, anElectromyography (EMG) sensor, an Electroencephalogram (EEG) sensor, anElectrocardiogram (ECG) sensor, an IR sensor, an ultrasound sensor, aniris sensor, a fingerprint sensor, etc. The sensor(s) 365 can furtherinclude a control circuit for controlling at least one of the sensorsincluded therein. The sensor(s) 365 can be used to determine anorientation and facing direction of the UE for 360-degree video. Any ofthese sensor(s) 365 may be located within the UE 300, within a headsetconfigured to hold the UE 300, or in both the headset and UE 300, forexample, in embodiments where the UE 300 includes a headset.

The touchscreen 350 can include a touch panel, a (digital) pen sensor, akey, or an ultrasonic input device. The touchscreen 350 can recognize,for example, a touch input in at least one scheme among a capacitivescheme, a pressure sensitive scheme, an infrared scheme, or anultrasonic scheme. The touchscreen 350 can also include a controlcircuit. In the capacitive scheme, the touchscreen 350 can recognizetouch or proximity. As described in more detail below, the UE mayinclude circuitry for and applications for editing or playing a360-degree video. Although FIG. 3 illustrates one example of UE 300,various changes may be made to FIG. 3. For example, various componentsin FIG. 3 could be combined, further subdivided, or omitted andadditional components could be added according to particular needs. As aparticular example, the processor 340 could be divided into multipleprocessors, such as one or more central processing units (CPUs) and oneor more graphics processing units (GPUs). Also, while FIG. 3 illustratesthe UE 300 configured as a mobile telephone, tablet, or smartphone, theUE 300 could be configured to operate as other types of mobile orstationary devices.

As described in more detail below, the UE 300 can be one example of anyof client devices 108-114 for the use of playing a single interactivevideo file or editing multiple videos to compile to the singleinteractive video file. The UE 300 can be used to remotely edit oraccess an editing tool on a server, such as server 104 of FIG. 1 orserver 200 of FIG. 2. Additionally, the UE 300 can be used to play asingle interactive video file comprising multiple 360-degree videos thatinclude interactive features, such as hotspots.

Although FIGS. 2 and 3 illustrate examples of devices in a computingsystem, various changes may be made to FIGS. 2 and 3. For example,various components in FIGS. 2 and 3 could be combined, furthersubdivided, or omitted and additional components could be addedaccording to particular needs. As a particular example, the mainprocessor 340 could be divided into multiple processors, such as one ormore central processing units (CPUs) and one or more graphics processingunits (GPUs). Also, while FIG. 3 illustrates the UE 300 configured as amobile telephone or smartphone, client devices could be configured tooperate as other types of mobile or stationary devices. In addition, aswith computing and communication networks, UEs and servers can come in awide variety of configurations, and FIGS. 2 and 3 do not limit thisdisclosure to any particular UE or server.

FIG. 4 illustrates a mapping 400 for a plurality of 360-degree videos inaccordance with an embodiment of this disclosure. The embodiment of themapping 400 illustrated in FIG. 4 is for illustration only. Mappings fora plurality of 360-degree videos can come in a wide variety ofconfigurations, and FIG. 4 does not limit the scope of this disclosureto any particular implementation of a mapping.

In FIG. 4, the spheres represent 360-degree videos 402-412. Each of the360-degree videos 402-412 can include a plurality of portions of videothat when combined, create the 360-degree video. Each 360-degree videoincludes horizontal and vertical lines for illustration purposes to showthe stitching of the different portions of the 360-degree video. Indifferent embodiments, the stitching can be done in a different manner,such as different size and shapes as those indicated in FIG. 4. In yetfurther embodiments, the 360-degree video may be a single video, ratherthan multiple stitched video portions. Additionally, as discussedherein, a 360-degree video can include a full video, a segment or clipor a video, any other user defined video length or portion of a video.Also, as discussed herein, a 360-degree video can be a video filecontaining the data used to create the 360-degree video in a videoplayer.

In an embodiment of this disclosure, a UE 300 can be illustrativelypositioned at the center of a 360-degree video. When using the UE 300 ina hand-held manner, portions of the 360-degree video can be displayed inthe UE 300 and the user can cause movement of the 360-degree video toshow different portions of the 360-degree video.

Each video 402-412 can be linked to one or more of the other videos402-412. For example, as shown in FIG. 4, video 402 is linked to videos404-408, videos 406-408 are further linked to video 410, and video 410is further linked to video 412. Each video can transition to any linkedvideos. A transition can be referred to a jump, or jumping from onevideo to another. The transition can be direct and can also includespecial effects such as, but not limited to, fading. When a transitionoccurs, the video player replaces the currently playing video with thetransitioned to video.

In some embodiments, one or more links can be bi-directional. That is,the videos on each end of the link can transition to each other. Forexample, if the link between videos 402 and 404 is bi-directional, thenthe video player can transition from video 402 to 404, and vice versa.Additionally, some links can be unidirectional. That is, the transitioncan only occur in one direction between linked videos. For example, ifthe link between videos 410 and 412 is unidirectional in the directionof video 412, then the transition can only occur from video 410 to video412. In different embodiments, different triggers can cause atransition. For example, the trigger can be a hotspot, the end of avideo, a period of inactivity, etc.

The different videos in the single interactive video file may be definedby a user through a user interface (UI) by adding and arranging thedifferent videos, clips, or segments in a desired order or arrangement.The videos may then be linked or connected with each other and withchild-segments as required by the single interactive video file.

Each video is composed of a video clip or segment and a list of linksthat can potentially be triggered through respective hotspots. Whentriggered, the links advance the story (i.e., currently played video) tothe target video or take a side-quest through a vignette.

In FIG. 4, if the video 402 is a starting point for the video, thenvideo 404 is a child-segment of video 402. In this case, because video404 is not linked to any other videos, video 404 is a vignette. As usedherein, segments and vignettes may be examples of videos. Vignettes,once played, always return to the parent-segment, and no additionallinks exists within a vignette. In this example, video 406 is also achild-segment of video 402, but is not a vignette because video 406 isfurther linked to, and a parent-segment of, video 410. Video 410 is achild-segment of video 406. Segments can be used to indicate any othervideo that is not a vignette, such as videos that include multiple pathsto different other videos.

FIG. 4 provides a hierarchy of segments and vignettes and assigns anindex to each video associated to the videos (e.g., segment orvignette). The order of traversal is based on the video ordering as theuser created the videos and does not reflect the order of the videos asthey will be viewed in the story, except for the first video. In oneexample embodiment, during video traversal, each segment and all thevignettes associated with that segment can be processed before moving onto the next segment and its vignettes. This ordering of segments andvignettes is also used as the serialization order for concatenating allthe video clips into one. As the process traverses the segments andvignettes, the duration of each video is added to a counter, and thecounter is used to calculate the start and end times of the clips asthey will appear in the concatenated video.

The result of the data compilation into the single interactive videofile may be an ordered array of start and end time offsets thatrepresent the video locations that will be in the single interactivevideo file. Additionally, each element of the array also contains a listof links, each with a list of hotspots that can be activated by theuser, and the index of the target segment or vignette to seek to whenthe links are actually activated. Several additional optional fields canindicate to the client how to behave when transitioning between videoclips or when returning from a vignette to the parent segment (such aspreserving orientation.).

Although FIG. 4 illustrates an example video hierarchy, various changesmay be made to FIG. 4. For example, different videos could be combined,further subdivided, or omitted and additional videos could be addedaccording to particular needs. As a particular example, video 412 mayloop back to video 402.

FIG. 5 illustrates a portion 502 of a 360-degree video with a hotspot504 in accordance with an embodiment of this disclosure. In FIG. 5, inan embodiment of this disclosure, a display of a UE can display orrender only a portion 502 of a 360-degree video. This portion is basedon the viewing angle, which can be set based on a size of a display orvideo content. In one embodiment of this disclosure, portion 502includes hotspot 504. The hotspot 504 can be any graphic or icon thatidentifies a hotspot. In one embodiment, the hotspot is overlaid over anarea related to the hotspot. For example, the hotspot 504 in FIG. 5 isoverlaid over an underneath opening of a playground. This opening isrelated to the hotspot because the hotspot takes a user to a new videoshowing the underneath portion of the playground. In other embodiments,the hotspot could be to the side of the area related to the hotspot,such as when the hotspot is an arrow, or elsewhere in the video.

In different embodiments, the hotspot shape can be spherical,rectangular, square, custom, based on an outline of an object within the360-degree video, etc. The hotspot 504 can cause seamless jumping to anext scene of a video, playing audio when a user looks at a particularobject or direction in the scene, or informational media overlays whenthe user looks certain directions. The hotspot can be triggered by auser selection, a user focusing on the hotspot for a period of time, asa reaction to another event, etc. Although FIG. 5 illustrates an examplehotspot, various changes may be made to FIG. 5. For example, additionalhotspots can be include, or no hotspots.

One or more embodiments of this disclosure recognize and take intoaccount that a photo tour can include only static 360-degree photos(i.e., no video). One or more embodiments of this disclosure provide asystem that uses video and is able to switch between several differenttypes of 360-degree videos rapidly by selecting or looking at hotspotsin the video.

Additionally, in one or more embodiments, a 360-degree interactivevideo-editing tool can be web based to allow for creation of a projectthat includes 360-degree video clips or entire 360-degree videos.Various embodiments of this disclosure provide the ability to specifyhotspots, transitions, vignettes, overlays, different events, videosynchronization options, 360-degree audio, orientation options, timeoffsets, all while editing the content from inside of the 360 videos.

The embodiments of this disclosure provide a solution for creation of aninteractive video. A user of the solution can set a hotspot using userinterfaces. The solution may utilize a video editing tool allowingcompilation of the videos and hotspots into one package. The package mayuse a format allowing quick switching between transitions and cancombine multiple videos into a single video with timestamps andadditional data.

FIG. 6 illustrates an exemplary video-editing tool 600 used to compile asingle interactive video file 602 from 360-degree videos 604 inaccordance with an embodiment of this disclosure. The video-editing tool600 illustrated in FIG. 6 is for illustration only. Video-editing tool600 can come in a wide variety of configurations, and FIG. 6 does notlimit the scope of this disclosure to any particular implementation of avideo-editing tool.

In FIG. 6, the video-editing tool 600 can be located on server 200 or UE300. The video-editing tool 600 can be used to define metadata 606within the single interactive video file 602. The metadata 606 caninclude a link 607, hotspot 609, and event trigger 608 in the singleinteractive video file 602. The link 607 is an association between twoor more videos, that when executed, allows a transition between twovideos. For example, a link 607 can bridge videos 402 and 404 as shownin FIG. 4. The hotspot 609 is an area, portion and/or object of a videothat can be selected or activated in order to execute the link 607. Forexample, the hotspot 609 can be hotspot 504 as shown in FIG. 5. Theevent trigger is an event that can occur that can also execute the link607. For example, the event trigger 608 could include a voice command,an ending of a video, an elapsed period of time, a period of inactivity,etc.

Each link 607 includes a target video 610, an execution list of hotspotsand even triggers, and a time window 612 in which the link 607 can beactivated. Additional information can also be included that can alterthe behavior of the video file 602 in different ways. The link 607 alsoincludes an execution list 614 to identify which hotspots and eventtriggers are associated with the link 607. When the link 607 isselected, whether though activation of a hotspot 609 or triggering of anevent trigger 608, a video player 616 transitions from the currentlyplaying video 618 to the target video 610.

In some embodiments, the hotspot 609 can be associated with multiplelinks that operate during different time windows. For example, a hotspotover a door may take you into a video for a room behind the door. Iftriggered during a first time window or range, the video of the room maybe different than if triggered during a second time window.

When a hotspot 609 is triggered the video player jumps to a vignette,while an offset 620 may start running. When the vignette ends, the videoplayer returns to the main video with a time offset corresponding to thetime viewing the vignette. Using the offsets 620 provides for theperception that the main video has been running while the vignette hasbeen playing. The offsets 620 can be based on the duration of thevignettes. Alternatively, the main video could pause while viewing thevignette. In yet another embodiment, the main video can utilize a timeoffset so when a hotspot is triggered, an offset will be added to thestart time providing for a perception that the main video and vignettewere running concurrently.

The video-editing tool 600 can also be used to compile the videos 604into a single interactive video file 602 that can be streamed,downloaded, and played on the video player 616 capable of playing360-degree videos. The video player 616 can use the hotspot 609 or eventtrigger 608 to trigger actions, such as, but not limited to, jumping toanother video seamlessly, playing additional audio tracks, imageoverlays, text overlays, transitions, control panels, etc. The compilingof the videos 604 can be done client side or on the server side, or evenonce the data arrives to the video player 616.

In one embodiment, all of the videos 604 in an interactive web basedproject and the metadata 606 that the user created by using the tool 600are compiled into a package, such as single interactive video file 602,that can be streamed or downloaded and played back by the 360-degreevideo player 616. The user can then interact and seamlessly jump ortransition using hotspots in the videos 604.

In one or more embodiments of this disclosure, the videos 604 cancontinue playing and even looping while the user chooses which hotspotto look at and/or select. Because of the continuous play, a user sensesimmersion into the video, allowing seamless video transition experiencesto be constructed.

In one or more embodiments, time may elapse in one or more videos,within the single interactive video file 602, that the user has leftwhile they are watching a specific video in the sequence. In otherexample embodiments, the videos not currently being watched may pausewhile waiting for a user to return. Various embodiments of thisdisclosure simplify the process of video editing for a content creator.A content creator can use a web-based or downloaded application/tool todraw hotspots, set transitions, etc. without having to code the metadata606 or work with video compiling and transcoding.

In one embodiment, the result of the data compilation into the singleinteractive video file 602 is a single video that includes all of thevideos (segments and vignettes) ordered one after another. An orderedarray can be included in the metadata 606 of start and end times thatrepresent the video locations of each of the videos 604 within thesingle video. In this example, the video player may only load the singlevideo in order to load all of the segments and vignettes.

Although FIG. 6 illustrates an example video editing tool, variouschanges may be made to FIG. 6. For example, the single interactive videofile 602 can be sent to the video player 616, or in another example, thefile 602 can be accessed remotely from a server. In another example, theconfiguration of the metadata 606 could include other information aboutthe videos. In yet another example, the link 607 may also include theactual hotspots or event triggers.

FIG. 7 illustrates a pictorial representation of a video-editing tool700 in accordance with an embodiment of this disclosure. Video-editingtool 700 can be one example of video-editing tool 600 as shown in FIG.6. Video-editing tool 700 includes adding segments 702 (i.e., videos),removing segments 704, setting hotspot time window 706, adjustingorientation 708, allowing an option to swipe 710 to go back to a priorsegment, adding a vignette 712 to any of the segments, selectinghotspots 714, etc.

The video-editing tool 700 can provide a project that includes theability to add or remove segments to a video. Each segment can be itsown video or a clip of a video. FIG. 7 represents a screenshot of thevideo-editing tool 700 at a particular moment of the video creationprocess. For example, in FIG. 7, New Segment 1 is currently beingedited. New Segment 1 includes Vignettes 0 and 1. Each of the vignettescan be selected to preserve an orientation through box 708 and allow theoption to swipe out of the vignette through box 710. Additionalvignettes can be added through button 712.

New Segment 1 can include hotspots, that when activated, triggerdifferent vignettes 0 or 1. These hotspots can be selected throughbutton 714. The hotspots for the vignettes can be shown on the NewSegment 1 during the specified time window 706. Each of the vignettescan include an option for transitioning or play through. When thetransitioning box is selected, the vignette automatically transitionsback to the New Segment 1 main video when the vignette is finished. Whenthe “through” box is selected, the vignette plays through and continuesto loop.

Although FIG. 7 illustrates an example screenshot of a video-editingtool 700, various changes may be made to FIG. 7. For example, checkboxes could be drop down boxes or other types of selection types. Inanother example, the hierarchy for segments and vignettes can bepresented in a different manner.

FIG. 8 illustrates a pictorial representation of selecting hotspots 800in video-editing tool 700 in accordance with an embodiment of thisdisclosure. When button 714 is selected as shown in FIG. 7, thevideo-editing tool 700 can include a visual user interface that showsthe main 360-degree video of New Segment 1. Different hotspots can beidentified through user selection of various points of interaction ortriggering. The hotspots may show up on an overlay of the video. Thebutton 802 can be selected to enter an edit mode to add, delete, orotherwise modify the hotspot. In one embodiment, multiple hotspots canbe added in the edit mode. In this embodiment, the hotspots added foreach vignette can be added separately.

Once hotspot locations are identified, further details can be determinedthrough a composer aspect of the interface. Once all the hotspots aredetermined, all the information, including additional video clips,metadata, hotspot locations, etc. may be submitted to a compiler. Thecompiler may be either on the client device or on a server.

Although FIG. 8 illustrates an example pictorial representation ofselecting hotspots 800 in video-editing tool 700, various changes may bemade to FIG. 8. For example, there may be an additional selection box toswitch between vignettes in order to edit hotspots from multiplevignettes without exiting this specific dialog.

FIG. 9 illustrates a process 900 for creating a single interactive videofile according to embodiments of the present disclosure. The process 900depicted in FIG. 9 may be performed by the UE 300 or differentcomponents of the UE 300 in FIG. 3, or the server 200 or differentcomponents of server 200 in FIG. 2.

At step 905, the process 900 begins with the server 200 receiving aselection of a plurality of 360-degree videos including a first360-degree video and a second 360-degree video. In differentembodiments, the videos can be uploaded from a UE or client device,retrieved from a website, created within the video editing tool of theserver 200, etc. While only first and second video are referenced here,additional videos may exist within the plurality of 360-degree videos.In one or more example embodiments, not all of the videos may be360-degree videos and some of the videos may be traditional videos whereall of the content is viewable at one time in a device.

At step 910, the server 200 can receive a selection of a portion of thefirst 360-degree video to form a hotspot. In one embodiment, the portioncan be selected by a user. In another embodiment, the server 200 canselect the portion based on video information. In this embodiment, theserver 200 can identify an object in the video and automatically createa hotspot. At step 915, the server 200 can receive an identification ofthe second 360-degree video for linking through the hotspot. In oneembodiment, the portion can be selected by a user. In anotherembodiment, the server 200 can select the portion based on videoinformation.

At step 920, the server 200 can link the first 360-degree video to thesecond 360-degree video to enable transitioning between the first360-degree video and the second 360-degree video. Based on theselections and identification of the videos and hotspots in steps 910and 915, the server 200 can create a link. When creating a link, theserver 200 includes metadata to include in a single interactive videofile that indicates the two linked videos, the linking activity(hotspot, event triggers, etc.), and any other relevant information(time window, offsets, etc.). The transitioning can occur in differentmanners. For example, in one embodiment, the transition can replace aportion of the first video with a new portion. In another example, theentire first video is replaced with the second vide.

At step 925, the server 200 can compile the plurality of 360-degreevideos into the single interactive file. The single interactive videofile includes an index used to identify the second 360-degree videowithin the single interactive video file when the link is executed. Themetadata information based on the information from step 920 can beincluded into the video file to identify the second video for the link.In this manner, the video file includes all used videos and metadata tolink different videos.

During steps 910-925, the server 200 can obtain the videos, concatenatethe videos in the right order, and transcode the file. The datacompilation produces a list of videos and the order in which the videoscan be compiled. If any segments contain vignettes, the time offsets ofthe segments and vignettes will be passed to the video transcoder tool(ffmpeg) so that more key frames can be inserted to improve video seekaccuracy and latency. The server 200 can start by downloading thenormalized form of the video assets. Each video can be normalized to thesame frame rate and resolution so that the ffmpeg muxer can process theminto a single video without strange artifacts caused by inconsistentvideo types. The resulting video is then inserted into the videotranscoding pipeline with optional time offsets where needed forimproved seek when jumping between vignettes and their parent segments.The transcoding may utilize region adaptive smoothing, fixed smoothing,or variable smoothing.

The resulting single interactive video file can be a streamableoptimized package. The 360-degree video transcoder may use varioussmoothing techniques and HEVC h.265 standard encoding to produce a moviefile as small as possible, and combine the individual videos all intoone segmented video. The single interactive video file allows for asmaller package that can be downloaded and streamed using lessbandwidth. In various embodiments, different encoding techniques may beused, such as, for example, x265.

At step 930, the server 200 can provide the single interactive videofile to the video player. The video player can exist on the server 200,another server, UE 300, or another UE. The file can be uploaded toanother device from the server 200, or accessed on the fly from theserver 200. The single interactive video file, during playback, candisplay interactable aspects (e.g., hotspots) based on various usersettings. The hotspots may be highlighted, overlaid, transparent, onlyshowing up if an indicator passes over them, etc.

In one or more embodiments, when providing the single interactive videofile to a video player, the server 200 may stream the file during a playback. When streaming, the server 200 may then use multiple video filesor stream each video within the file separately.

The single interactive video file includes all the videos with varioustimestamps or other metadata indicators to indicate which section of thevideo to jump to when a hotspot or trigger occurs. Such videos may notbe viewable without the trigger. This format allows for seamlessswitching. The seamless switching can be done in the interactive360-degree video player that reads the metadata along with the video.Because the compiler and transcoder put all of the individual videosinto a single interactive video file with short iframes (jump times inthe video) even mobile phones can be allowed to jump or seek points inthe video almost instantly.

Although FIG. 9 illustrates an example process for creating a 360-degreevideo, various changes could be made to FIG. 9. For example, while shownas a series of steps, various steps in each figure could overlap, occurin parallel, occur in a different order, or occur multiple times.Additionally, one or more steps can be optional. For example, when avideo does not include a hotspot, steps 910 and 915 may be omitted. Inanother example, the file may not be provided to the video player andstep 930 can be omitted.

Although the figures illustrate different examples of user equipment,various changes may be made to the figures. For example, the userequipment can include any number of each component in any suitablearrangement. In general, the figures do not limit the scope of thisdisclosure to any particular configuration(s). Moreover, while figuresillustrate operational environments in which various user equipmentfeatures disclosed in this patent document can be used, these featurescan be used in any other suitable system.

None of the description in this application should be read as implyingthat any particular element, step, or function is an essential elementthat must be included in the claim scope. The scope of patented subjectmatter is defined only by the claims. Moreover, none of the claims isintended to invoke 35 U.S.C. §112(f) unless the exact words “means for”are followed by a participle. Use of any other term, including withoutlimitation “mechanism,” “module,” “device,” “unit,” “component,”“element,” “member,” “apparatus,” “machine,” “system,” “processor,” or“controller,” within a claim is understood by the applicants to refer tostructures known to those skilled in the relevant art and is notintended to invoke 35 U.S.C. §112(f).

Although the present disclosure has been described with an exemplaryembodiment, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present disclosure encompasssuch changes and modifications as fall within the scope of the appendedclaims.

What is claimed is:
 1. A method for creating a single interactive videofile, the method comprising: receiving a selection of a plurality of360-degree videos including a first 360-degree video and a second360-degree video; linking the first 360-degree video to the second360-degree video to enable transitioning between the first 360-degreevideo and the second 360-degree video; and compiling the plurality of360-degree videos into the single interactive file, wherein the singleinteractive video file includes an index used to identify the second360-degree video within the single interactive video file when the linkis executed.
 2. The method of claim 1, wherein linking the first360-degree video to the second 360-degree video further comprises:receiving a selection of a portion of the first 360-degree video to forma hotspot; and receiving an identification of the second 360-degreevideo for linking through the hotspot.
 3. The method of claim 2, whereinthe hotspot, when triggered in the first 360-degree video, is configuredin the single interactive video file to enable a video player totransition from the first 360-degree video to the second 360-degreevideo.
 4. The method of claim 1, wherein compiling the plurality of360-degree videos into the single interactive video file comprises:providing, in the single interactive video file, an array of start andend time offsets representing locations along a timeline for each of theplurality of 360-degree videos.
 5. The method of claim 1, whereincompiling the plurality of 360-degree videos into the single interactivevideo file comprises: providing, in the single interactive video file, alist of links associated with hotspots and an index of the plurality of360-degree videos to seek when each link is executed.
 6. The method ofclaim 3, wherein the single interactive video file is configured toreturn to the first 360-degree video in which the hotspot is previouslytriggered, after the second 360-degree video is finished playing.
 7. Themethod of claim 1, wherein the single interactive video file isconfigured to allow a video player to adjust a time of the first360-degree video based on a duration of the second 360-degree video whenreturning to the first 360-degree video from the second 360-degreevideo.
 8. The method of claim 1, wherein the single interactive videofile is configured to allow a video player to start a timeline of thesecond 360-degree video only after transitioning to the second360-degree video.
 9. An apparatus for creating a single interactivevideo file, the apparatus comprising: a memory element configured tostore the single interactive video file; and at least one processorcoupled to the memory element, the at least one processor configured to:receive a selection of a plurality of 360-degree videos including afirst 360-degree video and a second 360-degree video; link the first360-degree video to the second 360-degree video to enable transitioningbetween the first 360-degree video and the second 360-degree video; andcompile the plurality of 360-degree videos into the single interactivefile, wherein the single interactive video file includes an index usedto identify the second 360-degree video within the single interactivevideo file when the link is executed.
 10. The apparatus of claim 9,wherein the at least one processor is further configured to: receive aselection of a portion of the first 360-degree video to form a hotspot;and receive an identification of the second 360-degree video for linkingthrough the hotspot.
 11. The apparatus of claim 10, wherein the hotspot,when triggered in the first 360-degree video, is configured in thesingle interactive video file to enable a video player to transitionfrom the first 360-degree video to the second 360-degree video.
 12. Theapparatus of claim 9, wherein the at least one processor is furtherconfigured to: provide, in the single interactive video file, an arrayof start and end time offsets representing locations along a timelinefor each of the plurality of 360-degree videos.
 13. The apparatus ofclaim 9, wherein the at least one processor is further configured to:provide, in the single interactive video file, a list of linksassociated with hotspots and an index of the plurality of 360-degreevideos to seek when each link is executed.
 14. The apparatus of claim11, wherein the single interactive video file is configured to return tothe first 360-degree video in which the hotspot is previously triggered,after the second 360-degree video is finished playing.
 15. The apparatusof claim 9, wherein the single interactive video file is configured toallow a video player to adjust a time of the first 360-degree videobased on a duration of the second 360-degree video when returning to thefirst 360-degree video from the second 360-degree video.
 16. Theapparatus of claim 9, wherein the single interactive video file isconfigured to allow a video player to start a timeline of the second360-degree video only after transitioning to the second 360-degreevideo.
 17. A non-transitory computer-readable medium comprisinginstructions stored in a memory for creating a single interactive videofile that, when executed by at least one processor, causes an electronicdevice to: receive a selection of a plurality of 360-degree videosincluding a first 360-degree video and a second 360-degree video; linkbetween the first 360-degree video to the second 360-degree video toenable transitioning between the first 360-degree video and the second360-degree video; and compile the plurality of 360-degree videos intothe single interactive file, wherein the single interactive video fileincludes an index used to identify the second 360-degree video withinthe single interactive video file when the link is executed.
 18. Thenon-transitory computer-readable medium of claim 17, further comprisinginstructions that, when executed by the at least one processor, causesthe electronic device to: receive a selection of a portion of the first360-degree video to form a hotspot; and receive an identification of thesecond 360-degree video for linking through the hotspot.
 19. Thenon-transitory computer-readable medium of claim 18, wherein thehotspot, when triggered in the first 360-degree video, is configured inthe single interactive video file to enable a video player to transitionfrom the first 360-degree video to the second 360-degree video.
 20. Thenon-transitory computer-readable medium of claim 17, further comprisinginstructions that, when executed by the at least one processor, causesthe electronic device to: provide, in the single interactive video file,an array of start and end time offsets representing locations along atimeline for each of the plurality of 360-degree videos.